Image adjustment method and associated image processing circuit

ABSTRACT

The present invention provides an image adjustment method and associated image processing circuit for performing the following operations upon each pixel of an image: obtaining R, G, B values and infrared ray (IR) value corresponding to the current pixel; generating multiple initial compensation parameters corresponding to the R, G, B values; generating an over-compensation parameter according to the R, G, B and IR values; comparing the over-compensation parameter with at least one threshold value to generate a compensation adjustment coefficient; and performing IR crosstalk compensation upon the image with the compensation adjustment coefficient.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to an image compensation technique, andmore particularly, to a technique for compensating the infrared (IR)crosstalk, which is suitable for RGB-IR sensor and capable ofautomatically calculating suitable IR crosstalk parameters for a sensor.

2. Description of the Prior Art

Considering the existence of IR component in the ambient light, IRpixels in the hybrid R, G, B and IR filter array are utilized to absorblight. However, the crosstalk effects may occur in the RGB-IR spectrum,as shown in FIG. 2. The occurrence of the overlaps in FIG. 2 is causedmainly by the interference between the IR light and the RGB light.Conventional sensor manufacturing techniques so far are still unable toblock or absorb non-color signals. Hence, when containing high IRcomponent of light energy, the color of the object can be influenced bythe IR crosstalk effect and thereby causes the color washout effectmight occur on the color of the image of an object, such as colorshifting. Further, the IR light may be oozed from a facial recognitionsystem of a video system, or from yellowish light (such as a halogenlamp).

The document US2010/0289885A1 discloses compensating the IR crosstalk bydeducting a certain proportion of IR value from each of the R, G, Bvalues, respectively. The equations are shown as follows:R _(new) =R _(ori) −k1×IR _(ori)G _(new) =G _(ori) −k2×IR _(ori)B _(new) =B _(ori) −k3×IR _(ori)IR _(new) =IR _(ori)wherein R_(new), G_(new), B_(new) and IR_(new) are the adjusted R, G, Bvalues and the IR value respectively, R_(ori), G_(ori), B_(ori) andIR_(ori) are the original R, G, B values and the IR value respectively,and k1, k2 and k3 are constants set by the user according to theinfluence of the IR crosstalk. This related art, however, merely roughlycancels a slight amount of the influence introduced by the infrared rayrather than adjusting based on the actual influence, and thus may causeover-compensation or distortion in R, G, B colors. Take the compensationequation “R__(new)=R__(ori)−k1*IR__(ori)” of the red light value as anexample, although this compensation equation in general provides acertain amount of compensation effect, when IR__(ori) is too large, thecompensation value will also increase, even making the value ofk1*IR__(ori) greater than R__(ori). In this case, the calculated R_(new)is obviously not correct, which is 0 or negative number). Thisphenomenon occurs more easily in an outdoor environment where thesunlight is strong or an indoor environment where the light emitted froma lamp contains a great portion of IR, making the IR crosstalkcompensation value too high and thereby causing color shifting. Moreparticularly, the image is more likely to be greenish.

Hence, there is a need for a novel method to solve the above problemwithout introducing side effects.

SUMMARY OF THE INVENTION

An objective of the present invention is to provide image adjustmentmethod and associated image processing circuit, in order to improve theaforementioned problem encountered in related art techniques.

An embodiment of the present invention provides an image adjustmentmethod for performing following operations upon each pixel of an image:obtaining R, G, B values and infrared ray (IR) value corresponding tothe current pixel; generating multiple initial compensation parameterscorresponding to the R, G, B values; generating an over-compensationparameter according to the R, G, B and IR values; comparing theover-compensation parameter with at least one threshold value togenerate a compensation adjustment coefficient; and performing IRcrosstalk compensation upon the image with the compensation adjustmentcoefficient.

An embodiment of the present invention provides an image processingcircuit which comprises a storage unit and a processor. The storage unitis arranged to temporarily store data. The processor is arranged toreceive an image and perform following operations upon each pixel of theimage: obtaining R, G, B values and infrared ray (IR) valuecorresponding to a current pixel; generating multiple initialcompensation parameters respectively corresponding to the R, G, Bvalues; generating an over-compensation parameter according to theinitial compensation parameters respectively corresponding to the R, G,B values and according to the IR value; comparing the over-compensationparameter with at least one threshold value, in order to generate acompensation adjustment coefficient; and using the compensationadjustment coefficient to perform IR crosstalk compensation upon theimage.

Embodiments of the present invention may detect over-compensated areasregion by region, and may apply a different adjustment parameter on eachover-compensated area, thus preventing the over-compensation to the IRcrosstalk.

These and other objectives of the present invention will no doubt becomeobvious to those of ordinary skill in the art after reading thefollowing detailed description of the preferred embodiment that isillustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart illustrating an image compensation methodaccording to an embodiment of the present invention.

FIG. 2 is a diagram illustrating an image processing circuitcorresponding to FIG. 1 according to an embodiment of the presentinvention.

DETAILED DESCRIPTION

Some phrases in the present specification and claims refer to specificelements; however, please note that the manufacturer might use differentterms to refer to the same elements. Further, in the presentspecification and claims, the term “comprising” is open type and shouldnot be viewed as the term “consists of.” The term “electrically coupled”can refer to either direct connection or indirect connection betweenelements. Thus, if the specification describes that a first device iselectrically coupled to a second device, the first device can bedirectly connected to the second device, or indirectly connected to thesecond device through other devices or means.

The present invention provides an IR crosstalk over-compensationpreventing mechanism to solve the aforementioned problem encountered inrelated art techniques. Techniques of the present invention may beroughly categorized as “Detection of possible over-compensated areas”and “Adjustment of compensation parameters for over-compensated areas”.The details are as follows.

I. Detection of Possible Over-Compensated Areas

Take the red light as example, since the over-compensation is caused bythe value of “k1*IR_ori” being extremely close to R_ori or even greaterthan R_ori, k1*(IR_ori/R_ori) is chosen as a candidate compensationcharacteristic value. This method may also be used to determine thecandidate compensation characteristic value corresponding to green lightand blue light. The over-compensation characteristic value X can bedetermined by choosing the largest one from the candidate compensationcharacteristic values corresponding to R, G, B as, The larger theover-compensation characteristic value is, the higher possibility thatthe corresponding pixel is an over-compensated pixel. Theover-compensation characteristic value X is calculated as the followingequation:

$X = {\max\left\{ \begin{matrix}{k1\left( {i,j} \right)*\frac{{IR}\left( {i,j} \right)}{{R\_ ori}\left( {i,j} \right)}} \\{k2\left( {i,j} \right)*\frac{{IR}\left( {i,j} \right)}{{G\_ ori}\left( {i,j} \right)}} \\{k3\left( {i,j} \right)*\frac{{IR}\left( {i,j} \right)}{{B\_ ori}\left( {i,j} \right)}}\end{matrix} \right.}$wherein k1, k2 and k3 are the initial compensation coefficientscorresponding to the R, G, B values respectively; IR(i, j), R_ori (i,j), G_ori(i, j) and B_ori(i, j) are primitive IR, R, G, B values of thecurrent pixel respectively; and k1(i, j), k2 (i, j) and k3 (i, j) are IRcrosstalk initial compensation parameters of R, G, B of the currentpixel respectively.II. Adjustment of Compensation Parameters for Over-Compensated Areas

This section illustrates the adjustment performed upon compensationparameters according to the over-compensation characteristic value X inorder to obtain the compensation adjustment coefficient W_final, whichmay be presented in the following equation:

${W\_ final} = \left\{ \begin{matrix}{{W\; 1\mspace{14mu}{or}\mspace{14mu} 1},{X < {{TH}1}}} \\{{{W1} + {\left( {{W2} - {W1}} \right)*\frac{X - {TH1}}{{TH2} - {TH1}}}}\ ,{{{TH}\ 1} < X < {TH2}}} \\{{W\; 2},{X > {{TH}2}}}\end{matrix} \right.$wherein TH1, TH2 are the first, second over-compensation thresholdvalues set by the user respectively, and W1, W2 are upper adjustmentlimit and lower adjustment limit set by the user respectively. Forexample, TH1, TH2 may be set as 0.8, 1.05 respectively, and W1, W2 maybe set as 0.8, 0.6 respectively, but this is merely one possibleparameter setting of the present invention and is not be a limitation ofthe present invention. In addition, the above equation may be achievedvia establishing a look-up table (LUT).

As can be seen from the first row of the aforementioned equation, whenthe over-compensation characteristic value X is lower than TH1 (e.g.lower than 0.8) , since the possibility such a value causesover-compensation is relatively lower, the upper limit of the W_finalvalue may be W1 or 1, that is to say, the adjustment may not beperformed, or be performed in a reduced extent. For example, when X islower than TH1, if there is no need for performing adjustment uponinitial compensation parameter, the value of W_final can be set as 1.This is because the last outputted final compensation parameter k_finalwill be equal to the initial compensation parameter k multiplied byW_final, making the value of the final compensation parameter k_finalremain at the value of the initial compensation parameter k. In anotherembodiment, when X is lower than TH1, the value of W_final can be W1 byonly adjusting the initial compensation parameter. This can limit thedifference between the final compensation parameter k_final and theinitial compensation parameter k.

Next, please refer to the second row of the equation, when theover-compensation characteristic value X is between TH1, TH2, the valueW will be adjusted in order to make the value of W_final exist betweenW1, W2, wherein the closer the X approaches TH1, the closer the W_finalwill approach W1. On the contrary, the closer the X approaches TH2, thecloser the W_final will approach W2. As long as similar effects can besubstantially achieved, the way of determining W_final can be modified.That is, the way of determining W_final does not necessarily need tofollow the interpolation relationship shown in the second row of theabove equation.

Next, please refer to the third row of the equation, whenover-compensation characteristic value Xi is larger than TH2, it meansthe current environment is under extremely high IR value, such as anenvironment under strong sunlight. In this situation, W_final will beadjusted downward but will not be no lower than W2, so that the pixelwill not be over downward-adjusted, in order to prevent from generatingcolor shifting effects.

After W_final is obtained, the initial compensation parametersrespectively corresponding to R, G, B are adjusted by being multipliedby the compensation adjustment coefficient W_final, in order to gainadjusted compensation parameters k1_final, k2_final and k3_finalrespectively corresponding to R, G, B. The associated equations arepresented as follows:k1_final=k1(i,j)*W_final;k2_final=k2(i,j)*W_final; andk3_final=k3(i,j)*W_final;wherein k1_final, k2_final and k3_final are the final compensationcoefficients respectively corresponding to R, G, B.

Next, the above final compensation parameters k1_final, k2_final andk3_final are used to perform IR crosstalk compensation. The associatedequations are as follows:R_final(i,j)=R_ori(i,j)−k1_final*IR(i,j);G_final(i,j)=G_ori(i,j)−k2_final*IR(i,j); andB_final(i,j)=B_ori(i,j)−k3_final*IR(i,j).wherein R_final (i, j), G_final (i, j) and B_final (i, j) are adjustedR, G, B values to be finally outputted, which may greatly improve thecolor shifting problem caused by over-compensation in conventionaltechniques, and may make the colors of the image more precise andrealistic.

In view of the above, the present invention provides a characteristicvalue calculation method for detecting IR crosstalk over-compensation,and an IR crosstalk over-compensation parameter adjustment method. Thepresent invention may effectively detect over-compensation areas in animage, and may apply a different adjustment parameter (e.g. a smallerone) on the over-compensation areas in order to prevent the IR crosstalkover-compensation effect. In addition, since each pixel of the image isperformed with its own IR value detection, the present invention iscapable of only adjusting the portions of the image where the IRcrosstalk is over-compensated, without altering the portions of theimage where the IR crosstalk is properly compensated or only mildlyadjusting these portions.

The aforementioned invention concepts may be concluded as the imagecompensation method 100 shown in FIG. 1. If the result is substantiallythe same, the steps are not required to be executed in the exact ordershown in FIG. 1. The flow shown in FIG. 1 may be adopted by the imageprocessing circuit 200 shown in FIG. 2, wherein the image processingcircuit 200 comprises a storage unit 210 and a processor 220, and theimage processing circuit 200 may be applied to various cameras and videodevices. The storage unit 210 is arranged to temporarily store data, andthe processor 220 is arranged to execute the image compensation method100 and perform various associated operations. The image compensationmethod 100 may be summarized as follows:

Step 102: Receive an image;

Step 104: Obtain R, G, B and IR values corresponding to the currentpixel;

Step 106: Generate initial compensation parameters respectivelycorresponding to the R, G, B values;

Step 108: Generate an over-compensation parameter according to theinitial compensation parameters corresponding to R, G, B values and theIR value;

Step 110: Compare the over-compensation parameter with a thresholdvalue, in order to generate a compensation adjustment coefficient;

Step 112: Use the compensation adjustment coefficient to perform IRcrosstalk compensation upon the image.

Those skilled in the art will readily observe that numerousmodifications and alterations of the device and method may be made whileretaining the teachings of the invention. Accordingly, the abovedisclosure should be construed as limited only by the metes and boundsof the appended claims.

What is claimed is:
 1. An image adjustment method for performingfollowing operations upon each pixel of an image: obtaining R, G, Bvalues and infrared ray (IR) value corresponding to the current pixel;generating multiple initial compensation parameters corresponding to theR, G, B values; generating an over-compensation parameter according tothe R, G, B and IR values; comparing the over-compensation parameterwith at least one threshold value to generate a compensation adjustmentcoefficient; and performing IR crosstalk compensation upon the imagewith the compensation adjustment coefficient.
 2. The image adjustmentmethod of claim 1, wherein the at least one threshold value comprises afirst over-compensation threshold value, a second over-compensationthreshold value, an upper adjustment limit and an lower adjustmentlimit; and the step of comparing the over-compensation parameter with atleast one threshold value to generate the compensation adjustmentcoefficient comprises: comparing the over-compensation parameter withthe over-compensation threshold value and the second over-compensationthreshold value, and obtaining the compensation adjustment coefficienthaving a value between the upper adjustment limit and the loweradjustment limit according to a comparison result.
 3. The imageadjustment method of claim 2, wherein the step of comparing theover-compensation parameter with the over-compensation threshold valueand the second over-compensation threshold value and obtaining thecompensation adjustment coefficient having the value between the upperadjustment limit and the lower adjustment limit according to thecomparison result is presented as the following equation:${W\_ final} = \left\{ \begin{matrix}{{W\; 1},{X < {{TH}1}}} \\{{{W1} + {\left( {{W2} - {W1}} \right)*\frac{X - {TH1}}{{TH2} - {TH1}}}}\ ,{{{TH}\ 1} < X < {TH2}}} \\{{W\; 2},{X > {{TH}2}}}\end{matrix} \right.$ wherein TH1, TH2 are the first over-compensationthreshold value and the second over-compensation threshold valuerespectively, the first over-compensation threshold value is not equalto the second over-compensation threshold value, W1, W2 are the upperadjustment limit and the lower adjustment limit respectively, X is theover-compensation parameter, and W_final is the compensation adjustmentcoefficient.
 4. The image adjustment method of claim 3, wherein thefirst over-compensation threshold value and the second over-compensationthreshold value are 0.8, 1.05 respectively, and the upper adjustmentlimit and the lower adjustment limit are 0.8, 0.6 respectively.
 5. Theimage adjustment method of claim 1, wherein the at least one thresholdvalue comprises a first over-compensation threshold value, a secondover-compensation threshold value, an upper adjustment limit and a loweradjustment limit; and the step of comparing the over-compensationparameter with at least one threshold value to generate the compensationadjustment coefficient comprises: comparing the over-compensationparameter with the over-compensation threshold value and the secondover-compensation threshold value, and referring to a comparison resultto obtain the compensation adjustment coefficient having the valuebetween 1 and the lower adjustment limit.
 6. The image adjustment methodof claim 5, wherein the step of comparing the over-compensationparameter with the over-compensation threshold value and the secondover-compensation threshold value and obtaining the compensationadjustment coefficient having the value between 1 and the loweradjustment limit according to comparison result is presented as thefollowing equation: ${W\_ final} = \left\{ \begin{matrix}{1,{X < {{TH}1}}} \\{{{W1} + {\left( {{W2} - {W1}} \right)*\frac{X - {TH1}}{{TH2} - {TH1}}}}\ ,{X < {{TH}2}}} \\{{W\; 2},{X > {{TH}2}}}\end{matrix} \right.$ wherein TH1, TH2 are the first over-compensationthreshold value and the second over-compensation threshold valuerespectively, the first over-compensation threshold value is not equalto the second over-compensation threshold value, W1, W2 are the upperadjustment limit and the lower adjustment limit respectively, X is theover-compensation parameter, and W_final is the compensation adjustmentcoefficient.
 7. The image adjustment method of claim 6, wherein thefirst over-compensation threshold value and the second over-compensationthreshold value are 0.8, 1.05 respectively, and the upper adjustmentlimit and the lower adjustment limit are 0.8, 0.6 respectively.
 8. Theimage adjustment method of claim 1, wherein the step of performing IRcrosstalk compensation upon the image with the compensation adjustmentcoefficient comprises: generate final compensation parametersrespectively corresponding to the R, G, B values according to thecompensation adjustment coefficient and the initial compensationparameters respectively corresponding to the R, G, B values, wherein thefinal compensation parameters corresponding to the R, G, B values arepresented by following equations:k1_final=k1(i,j)*W_final;k2_final=k2(i,j)*W_final; andk3_final=k3(i,j)*W_final; wherein k1_final, k2_final and k3_final arefinal compensation coefficients respectively corresponding to R, G, B,and W_final is the compensation adjustment coefficient.
 9. The imageadjustment method of claim 8, wherein the step of performing IRcrosstalk compensation upon the image with the compensation adjustmentcoefficient further comprises: respectively using the final compensationcoefficients corresponding to R, G, B to generate final output R, G, Bvalues corresponding to R, G, B, wherein the final output R, G, B valuesare presented by following equations:R_final(i,j)=R_ori(i,j)−k1_final*IR(i,j);G_final(i,j)=G_ori(i,j)−k2_final*IR(i,j); andB_final(i,j)=B_ori(i,j)−k3_final*IR(i,j); wherein R_final(i, j),G_final(i, j) and B_final(i, j) are final output R, G, B valuesrespectively, R_ori(i, j), G_ori(i, j), B_ori(i, j) and IR (i, j) arethe R, G, B and IR values respectively, and (i, j) is the coordinate ofa pixel.
 10. An image processing circuit, comprising: a processor,arranged to receive an image and perform following operations upon eachpixel of the image: obtaining R, G, B values and infrared ray (IR) valuecorresponding to a current pixel; generating multiple initialcompensation parameters respectively corresponding to the R, G, Bvalues; generating an over-compensation parameter according to theinitial compensation parameters respectively corresponding to the R, G,B values and according to the IR value; comparing the over-compensationparameter with at least one threshold value, in order to generate acompensation adjustment coefficient; and using the compensationadjustment coefficient to perform IR crosstalk compensation upon theimage.
 11. The image processing circuit of claim 10, wherein the atleast one threshold value comprises a first over-compensation thresholdvalue, a second over-compensation threshold value, an upper adjustmentlimit and an lower adjustment limit; and the step of comparing theover-compensation parameter with at least one threshold value in orderto generate the compensation adjustment coefficient comprises: comparingthe over-compensation parameter with the over-compensation thresholdvalue and the second over-compensation threshold value, and according toa comparison result, obtaining the compensation adjustment coefficientwith the size existing between the upper adjustment limit and the loweradjustment limit.
 12. The image processing circuit of claim 11, whereinthe step of comparing the over-compensation parameter with theover-compensation threshold value and the second over-compensationthreshold value and obtaining the compensation adjustment coefficientwith the size existing between the upper adjustment limit and the loweradjustment limit according to the comparison result is presented infollowing equation: ${W\_ final} = \left\{ \begin{matrix}{{W\; 1},{X < {{TH}1}}} \\{{{W1} + {\left( {{W2} - {W1}} \right)*\frac{X - {TH1}}{{TH2} - {TH1}}}}\ ,{{{TH}\ 1} < X < {TH2}}} \\{{W\; 2},{X > {{TH}2}}}\end{matrix} \right.$ wherein TH1, TH2 are the first over-compensationthreshold value and the second over-compensation threshold valuerespectively, the first over-compensation threshold value is not equalto the second over-compensation threshold value, W1, W2 are the upperadjustment limit and the lower adjustment limit, respectively, X is theover-compensation parameter, and W_final is the compensation adjustmentcoefficient.
 13. The image processing circuit of claim 12, wherein thefirst over-compensation threshold value and the second over-compensationthreshold value are 0.8 and 1.05 respectively, and the upper adjustmentlimit and the lower adjustment limit are 0.8 and 0.6 respectively. 14.The image processing circuit of claim 10, wherein the at least onethreshold value comprises a first over-compensation threshold value, asecond over-compensation threshold value, an upper adjustment limit andan lower adjustment limit; and the step of comparing theover-compensation parameter with at least one threshold value in orderto generate a compensation adjustment coefficient comprises: comparingthe over-compensation parameter with the over-compensation thresholdvalue and the second over-compensation threshold value, and referring toa comparison result to obtain the compensation adjustment coefficientwith the size existing between 1 and the lower adjustment limit.
 15. Theimage processing circuit of claim 14, wherein the step of comparing theover-compensation parameter with the over-compensation threshold valueand the second over-compensation threshold value and obtaining thecompensation adjustment coefficient with the size existing between 1 andthe lower adjustment limit according to the comparison result ispresented by the following equation:${W\_ final} = \left\{ \begin{matrix}{1,{X < {{TH}1}}} \\{{{W1} + {\left( {{W2} - {W1}} \right)*\frac{X - {TH1}}{{TH2} - {TH1}}}}\ ,{X < {{TH}2}}} \\{{W\; 2},{X > {{TH}2}}}\end{matrix} \right.$ wherein TH1, TH2 are the first over-compensationthreshold value and the second over-compensation threshold valuerespectively, the first over-compensation threshold value is not equalto the second over-compensation threshold value, W1, W2 are the upperadjustment limit and the lower adjustment limit respectively, X is theover-compensation parameter, and W_final is the compensation adjustmentcoefficient.
 16. The image processing circuit of claim 15, wherein thefirst over-compensation threshold value and the second over-compensationthreshold value are 0.8 and 1.05 respectively, and the upper adjustmentlimit and the lower adjustment limit are 0.8 and 0.6 respectively. 17.The image processing circuit of claim 10, wherein the step of using thecompensation adjustment coefficient to perform IR crosstalk compensationupon the image comprises: generating final compensation parametersrespectively corresponding to the R, G, B values according to thecompensation adjustment coefficient and the initial compensationparameters respectively corresponding to the R, G, B values, wherein thefinal compensation parameters corresponding to the R, G, B values arepresented by following equations:k1_final=k1(i,j)*W_final;k 2_final=k2(i,j)*W_final; andk 3_final=k3(i,j)*W_final; wherein k1_final, k2_final and k3_final arefinal compensation coefficients respectively corresponding to R, G, B,and W_final is the compensation adjustment coefficient.
 18. The imageprocessing circuit of claim 17, wherein the step of using thecompensation adjustment coefficient to perform IR crosstalk compensationupon the image comprises: respectively using the final compensationcoefficients corresponding to R, G, B to generate final output R, G, Bvalues corresponding to R, G, B, wherein the final output R, G, B valuesare presented by following equations:R_final(i,j)=R_ori(i, j)−k1_final*IR(i,j);G_final(i,j)=G_ori(i, j)−k2_final*IR(i,j); andB_final(i,j)=B_ori(i, j)−k3_final*IR(i,j); wherein R_final(i, j),G_final(i, j) and B_final (i, j) are final output R, G, B valuesrespectively, R_ori(i, j), G_ori(i, j), B_ori(i, j) and IR (i, j) arethe R, G, B and IR values respectively, and (i, j) is the coordinate ofa pixel.